Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

convert 'None' and '' string values to NoneType before forwarding the… #88

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

T0MASD
Copy link

@T0MASD T0MASD commented May 23, 2017

We had some issues importing logs that contained "None" string values, converting them NoneTypes did the trick.
[2017-05-22 03:41:43,092][DEBUG ][action.bulk ] [logstash-2017.05.22][2] failed to execute bulk item (index) index { [logstash-2017.05.22][fluentd][AVwuPw0zO--t2Sy8RmXq],source[{ "Status": "None", ...
Caused by: org.elasticsearch.index.mapper.MapperParsingException: object mapping for [fluentd] tried to parse field [status] as object, but got EOF, has a concrete value been provided to it?
I'm not sure where it's best to place the nonevalues=['None',''] to be used in _add_dic

@coveralls
Copy link

Coverage Status

Coverage increased (+0.04%) to 90.865% when pulling b290dbd on T0MASD:convert_none_strings into 1c69535 on fluent:master.

1 similar comment
@coveralls
Copy link

Coverage Status

Coverage increased (+0.04%) to 90.865% when pulling b290dbd on T0MASD:convert_none_strings into 1c69535 on fluent:master.

@T0MASD T0MASD force-pushed the convert_none_strings branch from b290dbd to 7b1a26d Compare May 23, 2017 03:31
@coveralls
Copy link

Coverage Status

Coverage increased (+0.04%) to 90.865% when pulling 7b1a26d on T0MASD:convert_none_strings into 1c69535 on fluent:master.

3 similar comments
@coveralls
Copy link

Coverage Status

Coverage increased (+0.04%) to 90.865% when pulling 7b1a26d on T0MASD:convert_none_strings into 1c69535 on fluent:master.

@coveralls
Copy link

Coverage Status

Coverage increased (+0.04%) to 90.865% when pulling 7b1a26d on T0MASD:convert_none_strings into 1c69535 on fluent:master.

@coveralls
Copy link

Coverage Status

Coverage increased (+0.04%) to 90.865% when pulling 7b1a26d on T0MASD:convert_none_strings into 1c69535 on fluent:master.

@repeatedly
Copy link
Member

Sorry for the late reply.
How about setting this values via class variable like below?

fluent.handler.FluentHandler.NoneValues = ["None"] # default is None

Changing default behaviour may break existing environment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants